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Abstract 

Causality is typically treated an all-or-nothing concept; either ^ is a cause of 
B or it is not. We extend the definition of causality introduced by Halpern and 
Pearl [2001a] to take into account the degree of responsibility of A for B. For 
example, if someone wins an election 11~0, then each person who votes for him is 
less responsiblefor the victory than if he had won 6-5. We then define a notion 
of degree of blame, which takes into account an agent's epistemic state. Roughly 
speaking, the degree of blame of A for B is the expected degree of responsibility of 
A for B, taken over the epistemic state of an agent. 
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1 Introduction 



There have been many attempts to define causality going back to Hume [1739], and 
continuing to the present (see, for example, [Colhns, Hall, and Paul 2003; Pearl 2000] for 
some recent work). While many definitions of causality have been proposed, all of them 
treat causahty is treated as an all-or-nothing concept. That is, A is either a cause of B or 
it is not. As a consequence, thinking only in terms of causality does not at times allow us 
to make distinctions that we may want to make. For example, suppose that Mr. B wins 
an election against Mr. G by a vote of 11-0. Each of the people who voted for Mr. B is 
a cause of him winning. However, it seems that their degree of responsibility should not 
be as great as in the case when Mr. B wins 6-5. 

In this paper, we present a definition of responsibility that takes this distinction into 
account. The definition is an extension of a definition of causality introduced by Halpern 
and Pearl [2001a]. Like many other definitions of causality going back to Hume [1739], 
this definition is based on counterfactual dependence. Roughly speaking, A is a. cause 
of B if, had A not happened (this is the counterfactual condition, since A did in fact 
happen) then B would not have happened. As is well known, this naive definition does 
not capture all the subtleties involved with causality. (If it did, there would be far fewer 
papers in the philosophy literature!) In the case of the 6-5 vote, it is clear that, according 
to this definition, each of the voters for Mr. B is a cause of him winning, since if they had 
voted against Mr. B, he would have lost. On the other hand, in the case of the 11-0 vote, 
there are no causes according to the naive counterfactual definition. A change of one vote 
does not makes no difference. Indeed, in this case, we do say in natural language that the 
cause is somewhat "diffuse". 

While in this case the standard counterfactual definition may not seem quite so prob- 
lematic, the following example (taken from [Hall 2003]) shows that things can be even 
more subtle. Suppose that Suzy and Billy both pick up rocks and throw them at a bottle. 
Suzy's rock gets there first, shattering the bottle. Since both throws are perfectly accu- 
rate, Billy's would have shattered the bottle had Suzy not thrown. Thus, according to 
the naive counterfactual definition, Suzy's throw is not a cause of the bottle shattering. 
This certainly seems counter to intuition. 

Both problems are dealt with the same way in [Halpern and Pearl 2001a]. Roughly 
speaking, the idea is that ^4 is a cause of S if S counterfactually depends on C under some 
contingency. For example, voter 1 is a cause of Mr. B winning even if the vote is 11-0 
because, under the contingency that 5 of the other voters had voted for Mr. G instead, 
voter I's vote would have become critical; if he had then changed his vote, Mr. B would 
not have won. Similarly, Suzy's throw is the cause of the bottle shattering because the 
bottle shattering counterfactually depends on Suzy's throw, under the contingency that 
Billy doesn't throw. (There are further subtleties in the definition that guarantee that, 
if things are modeled appropriately, Billy's throw is not a cause. These are discussed in 
Section 2.) 

It is precisely this consideration of contingencies that lets us define degree of respon- 
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sibility. We take the degree of responsibility of A for B to be 1/{N +1), where N is the 
minimal number of changes that have to be made to obtain a contingency where B coun- 
tcrfactually depends on A. (If A is not a cause of B, then the degree of responsibility is 0.) 
In particular, this means that in the case of the 11-0 vote, the degree of responsibility of 
any voter for the victory is 1/6, since 5 changes have to be made before a vote is critical. 
If the vote were 1001-0, the degree of responsibility of any voter would be 1/501. On the 
other hand, if the vote is 5-4, then the degree of responsibility of each voter for Mr. B for 
Mr. B's victory is 1; each voter is critical. As we would expect, those voters who voted 
for Mr. G have degree of responsibility for Mr. B's victory, since they are not causes of 
the victory. Finally, in the case of Suzy and Billy, even though Suzy is the only cause of 
the bottle shattering, Suzy's degree of responsibihty is 1/2, while Billy's is 0. Thus, the 
degree of responsibility measures to some extent whether or not there are other potential 
causes. 

When determining responsibility, it is assumed that everything relevant about the 
facts of the world and how the world works (which we characterize in terms of what are 
called structural equations) is known. For example, when saying that voter 1 has degree 
of responsibility 1/6 for Mr. B's win when the vote is 11-0, we assume that the vote and 
the procedure for determining a winner (majority wins) is known. There is no uncertainty 
about this. Just as with causality, there is no difficulty in talking about the probability 
that someone has a certain degree of responsibility by putting a probability distribution on 
the way the world could be and how it works. But this misses out on important component 
of determining what we call here blame: the epistemic state. Consider a doctor who treats 
a patient with a particular drug resulting in the patient's death. The doctor's treatment 
is a cause of the patient's death; indeed, the doctor may well bear degree of responsibility 
1 for the death. However, if the doctor had no idea that the treatment had adverse side 
effects for people with high blood pressure, he should perhaps not be held to blame for the 
death. Actually, in legal arguments, it may not be so relevant what the doctor actually 
did or did not know, but what he should have known. Thus, rather than considering the 
doctor's actual epistemic state, it may be more important to consider what his epistemic 
state should have been. But, in any case, if we are trying to determine whether the doctor 
is to blame for the patient's death, we must take into account the doctor's epistemic state. 

We present a definition of blame that considers whether agent a performing action b 
is to blame for an outcome (p. The definition is relative to an epistemic state for a, which 
is taken, roughly speaking, to be a set of situations before action b is performed, together 
with a probability on them. The degree of blame is then essentially the expected degree 
of responsibility of action b for ip (except that we ignore situations where (p was already 
true or b was already performed) . To understand the difference between responsibility and 
blame, suppose that there is a firing squad consisting of ten excellent marksmen. Only 
one of them has live bullets in his rifle; the rest have blanks. The marksmen do not know 
which of them has the live bullets. The marksmen shoot at the prisoner and he dies. The 
only marksman that is the cause of the prisoner's death is the one with the live bullets. 
That marksman has degree of responsibility 1 for the death; all the rest have degree of 
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responsibility 0. However, each of the marksmen has degree of blame 1/10.^ 

While we believe that our definitions of responsibility and blame are reasonable, they 
certainly do not capture all the connotations of these words as used in the literature. 
In the philosophy literature, papers on responsibility typically are concerned with moral 
responsibility (see, for example, [Zimmerman 1988]). Our definitions, by design, do not 
take into account intentions or possible alternative actions, both of which seem necessary 
in dealing with moral issues. For example, there is no question that Truman was in 
part responsible and to blame for the deaths resulting from dropping the atom bombs 
on Hiroshima and Nagasaki. However, to decide whether this is a morally reprehensible 
act, it is also necessary to consider the alternative actions he could have performed, and 
their possible outcomes. While our definitions do not directly address these moral issues, 
we believe that they may be helpful in elucidating them. Shafer [2001] discusses a notion 
of responsibility that seems somewhat in the spirit of our notion of blame, especially in 
that he views responsibility as being based (in part) on causality. However, he does not 
give a formal definition of responsibility, so it is hard to compare our definitions to his. 
However, there are some significant technical differences between his notion of causality 
(discussed in [Shafer 1996]) and that on which our notions are based. We suspect that 
any notion of responsibility or blame that he would define would be different from ours. 
We return to these issues in Section 5. 

The rest of this paper is organized as follows. In Section 2 we review the basic defini- 
tions of causal models based on structural equations, which are the basis for our defini- 
tions of responsibility and blame. In Section 3, we review the definition of causality from 
[Halpcrn and Pearl 2001a], and show how it can be modified to give a definition of respon- 
sibility. We show that the definition of responsibility gives reasonable answer in a number 
of cases, and briefly discuss how it can be used in program verification (see [Chockler, 
Halpern, and Kupferman 2003]). In Section 3.3, we give our definition of blame. In Sec- 
tion 4, we discuss the complexity of computing responsibility and blame. We conclude in 
Section 5 with some discussion of responsibility and blame. 

2 Causal Models: A Review 

In this section, we review the details of the definitions of causal models from [Halpern 

and Pearl 2001a]. This section is essentially identical to the corresponding section in 
[Chockler, Halpern, and Kupferman 2003]; the material is largely taken from [Halpern 
and Pearl 2001a]. 

A signature is a tuple S = {U,V,7l), where W is a finite set of exogenous variables, 
V is a set of endogenous variables, and TZ associates with every variable Y & U U V a 
nonempty set T^iY) of possible values for Y. Intuitively, the exogenous variables are ones 
whose values are determined by factors outside the model, while the endogenous variables 

^We thank Tim Williamson for this example. 



3 



are ones whose values are ultimately determined by the exogenous variables. A causal 
model over signature iS is a tuple M = {S, JF), where JF associates with every endogenous 
variable X G V a function Fx such that Fx : {xu<zuTZ{U) x (xygv\{x}'^(^))) ^ 71{X). 
That is, Fx describes how the value of the endogenous variable X is determined by the 
values of all other variables in UUV. If TZiY) contains only two values for each Y e WU V, 
then we say that M is a binary causal model. 

We can describe (some salient features of) a causal model M using a causal network. 
This is a graph with nodes corresponding to the random variables in V and an edge from 
a node labeled X to one labeled Y if Fy depends on the value of X. Intuitively, variables 
can have a causal effect only on their descendants in the causal network; if Y is not a 
descendant of X, then a change in the value of X has no affect on the value of Y. For 
ease of exposition, we restrict attention to what are called recursive models. These are 
ones whose associated causal network is a directed acyclic graph (that is, a graph that 
has no cycle of edges). Actually, it suffices for our purposes that, for each setting u for 
the variables in U, there is no cycle among the edges of causal network. We call a setting 
u for the variables in a context. It should be clear that if M is a recursive causal model, 
then there is always a unique solution to the equations in M, given a context. 

The equations determined by {Fx : X E V} can be thought of as representing 
processes (or mechanisms) by which values are assigned to variables. For example, if 
Fx{Y, Z,U) = Y + U (which we usually write as X = F + f/), then if F = 3 and f/ = 2, 
then X = 5, regardless of how Z is set. This equation also gives counterfactual informa- 
tion. It says that, in the context U = 4, if y were 4, then X would he u + A, regardless 
of what value X, Y, and Z actually take in the real world. 

While the equations for a given problem are typically obvious, the choice of variables 
may not be. For example, consider the rock-throwing example from the introduction. In 
this case, a naive model might have an exogenous variable U that encapsulates whatever 
background factors cause Suzy and Billy to decide to throw the rock (the details of U do 
not matter, since we are interested only in the context where [/'s value is such that both 
Suzy and Billy throw), a variable ^Tfor Suzy throws {ST = 1 if Suzy throws, and ST = 
if she doesn't), a variable BT for Billy throws, and a variable BS for bottle shatters. In 
the naive model, whose graph is given in Figure 1, BS is 1 if one of ST and BT is 1. 
(Note that the graph omits the exogenous variable U, since it plays no role. In the graph, 
there is an arrow from variable X to variable Y if the value of Y depends on the value of 



This causal model does not distinguish between Suzy and Billy's rocks hitting the 
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Figure 1: A naive model for the rock-throwing example. 
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bottle simultaneously and Suzy's rock hitting first. A more sophisticated model might 
also include variables SH and BH, for Suzy's rock hits the bottle and Billy's rock hits the 
bottle. Clearly BS is 1 iff one of SH and BH is 1. However, now, SH is 1 if ST is 1, and 
BH = 1 if BT = 1 and SH = 0. Thus, Billy's throw hits if Billy throws and Suzy's rock 
doesn't hit. This model is described by the following graph, where we implicitly assume 
a context where Suzy throws first, so there is an edge from SH to BH, but not one in the 
other direction. 




Figure 2: A better model for the rock-throwing example. 



— * 

Given a causal model M — {S,T), a (possibly empty) vector X of variables in V, 

— * 

and vectors x and u of values for the variables in X and U, respectively, we can define a 

— * 

new causal model denoted Mj^^^ over the signature 5^ = {U, V — X, lZ\y_ji). Formally, 
^x^x ~ i^xT where Fy^^ is obtained from Fy by setting the values of the 

— * 

variables in X to x. Intuitively, this is the causal model that results when the variables 
in X are set to x by some external action that affects only the variables in X; we do 
not model the action or its causes exphcitly. For example, if M is the more sophisticated 
model for the rock-throwing example, then Mst^o is the model where Suzy doesn't throw. 

Given a signature S — {U,V,TZ), a formula of the form X — x, lor X & V and 
X G TZ{X), is called a primitive event. A basic causal formula has the form [Yi <— 
yi, . . . ^ where 

• </? is a Boolean combination of primitive events; 

• 1^1, . . . , Yfc are distinct variables in V; and 

• Vi e n{Yi). 

Such a formula is abbreviated as [y <— y]</?. The special case where A; = is abbreviated 
as if. Intuitively, \Vi ^ yi, . . . , Y^. ^ UkW says that (p holds in the counterfactual world 
that would arise if Yi is set to i = 1, . . . , A;. A causal formula is a Boolean combination 
of basic causal formulas. 

A causal formula 93 is true or false in a causal model, given a context. We write 
(M, u) 1= 99 if 93 is true in causal model M given context u. (M, u) |= [F <— ^ (X = x) 
if the variable X has value x in the unique (since we are dealing with recursive models) 
solution to the equations in M^^^ in context u (that is, the unique vector of values for 

the exogenous variables that simultaneously satisfies all equations F^^^ , Z G V — Y , with 
the variables in U set to u) . We extend the definition to arbitrary causal formulas in the 
obvious way. 
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3 Causality and Responsibility 



3.1 Causality 

We start with the definition of cause from [Halpern and Pearl 2001a]. 

Definition 3.1 (Cause) We say that X — x is a cause of (p in (M, u) if the following 
three conditions hold: 

ACl. {M,u) ^{X = x)A(p. 

AC2. There exist a partition {Z,W) of V with X C Z and some setting {x',w') of the 
variables in {X, W) such that if (M, u) \= Z = z* for Z E Z, then 

— * — * — * — * 

(a) {M,u) \= [X a;', 'w']-'ip. That is, changing {X,W) from {xjw) to 
{x',w') changes ip from true to false. 

(b) (M, u) \= [X ^ x,W ^ w', Z' ^ z*](p for all subsets Z' of Z. That is, setting 

— * — * 

W to w' should have no effect on (p as long as X has the value x, even if all 
the variables in an arbitrary subset of Z are set to their original values in the 
context u. 

ACS. {X — x) is minimal, that is, no subset of X satisfies AC2. 

ACl just says that A cannot be a cause of B unless both A and B are true, while ACS 
is a minimality condition to prevent, for example, Suzy throwing the rock and sneezing 
from being a cause of the bottle shattering. Eiter and Lukasiewicz [2002b] showed that 
one consequence of ACS is that causes can always be taken to be single conjuncts. Thus, 
from here on in, we talk about X = x being the cause of (/?, rather than X = x being the 
cause. The core of this definition lies in AC2. Informally, the variables in Z should be 
thought of as describing the "active causal process" from X to </?. These are the variables 
that mediate between X and p. AC2(a) is reminiscent of the traditional counterfactual 
criterion, according to which A = x is a cause of tp if change the value of X results in p 
being false. However, AC2(a) is more permissive than the traditional criterion; it allows 
the dependence of <^ on A to be tested under special structural contingencies, in which 
the variables W are held constant at some setting w' . AC2(b) is an attempt to counteract 
the "permissiveness" of AC2(a) with regard to structural contingencies. Essentially, it 
ensures that X alone suffices to bring about the change from p to -^p\ setting W to w' 
merely eliminates spurious side effects that tend to mask the action of X. 

To understand the role of AC2(b), consider the rock-throwing example again. In 
the model in Figure 1, it is easy to see that both Suzy and Billy are causes of the bottle 
shattering. Taking Z — {ST, BS}, consider the structural contingency where Billy doesn't 
throw (BT^O). Clearly [ST ^ 0, BT ^ 0]BS = and [ST ^ 1, BT ^ 0]BS = 1 both 
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hold, so Suzy is a cause of the bottle shattering. A symmetric argument shows that Billy 
is also a cause. 

But now consider the model described in Figure 2. It is still the case that Suzy is a 
cause in this model. We can take Z = {ST, SH, BS} and again consider the contingency 
where Billy doesn't throw. However, Billy is not a cause of the bottle shattering. For 
suppose that we now take Z — {BT, BE, BS} and consider the contingency where Suzy 
doesn't throw. Clearly AC2(a) holds, since if Billy doesn't throw (under this contingency), 
then the bottle doesn't shatter. However, AC2(b) does not hold. Since BH G Z, if we set 
BH to (it's original value), then AC2(b) requires that [BT^ 1, ST ^ 0, BH ^ 0]{BS = 
1) hold, but it does not. Similar arguments show that no other choice of {Z, W) makes 
Billy's throw a cause. 

In [Halpern and Pearl 2001a], a slightly more refined definition of causahty is also 
considered, where there is a set of allowable settings for the endogenous settings, and 
the only contingencies that can be considered in AC2(b) are ones where the settings 
{W = w', X = x') and {W = w', X = x) are allowable. The intuition here is that we do 
not want to have causality demonstrated by an "unreasonable" contingency. For example, 
in the rock throwing example, we may not want to allow a setting where ST = 0, BT = 1, 
and BH — 0, since this means that Suzy doesn't throw, Billy does, and yet Billy doesn't hit 
the bottle; this setting contradicts the assumption that Billy's throw is perfectly accurate. 
We return to this point later. 

3.2 Responsibility 

The definition of responsibihty in causal models extends the definition of causahty. 

Definition 3.2 (Degree of Responsibility) The degree of responsibility of X = x for 

ip in (M, -u), denoted dr((M, -u), (X = x),ip), is if X = x is not a cause of Lp in {M,u); 
it is l/{k + 1) if X = x is a cause of ip in (M, u) and there exists a partition {Z, W) and 

— * 

setting {x', w') for which ACS holds such that (a) k variables in W have different values in 

— * — * 

w' than they do in the conteod u and (b) there is no partition (Z', W') and setting {x", w") 
satisfying AC2 such that only k' < k variables have different values in w" than they do 
the context u. 

Intuitively, dr{{M, m), (X = x), tp) measures the minimal number of changes that have 
to be made in u in order to make (/? counterf actually depend on X. If no partition of 
V to {Z,W) makes ip counterfactually depend on (X = x), then the minimal number 
of changes in u in Definition 3.2 is taken to have cardinality oo, and thus the degree of 
responsibility of X = x is 0. U ip counterfactually depends on X = x, that is, changing 
the value of X alone falsifies (p in (M, u), then the degree of responsibility of X = a; in 
(/? is 1. In other cases the degree of responsibihty is strictly between and 1. Note that 
X — X is a cause of (p iff the degree of responsibility oi X — x ior (p is greater than 0. 
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Example 3.3 Consider the voting example from the introduction. Suppose there are 11 

voters. Voter i is represented by a variable Xj, i = the outcome is represented 

by the variable O, which is 1 if Mr. B wins and if Mr. B wins. In the context where 
Mr. B wins 11-0, it is easy to check that each voter is a cause of the victory (that is 
Xj = 1 is a cause of O = 1, for i = 1, . . . , 11). However, the degree of responsibihty of 
Xj = 1 for is O = 1 is just 1/6, since at least five other voters must change their votes 
before changing Xj to results in O = 0. But now consider the context where Mr. B wins 
6-5. Again, each voter who votes for Mr. B is a cause of him winning. However, now 
each of these voters have degree of responsibility 1. That is, if Xj = 1, changing Xj to 
is already enough to make O = 0; no other variables need to change. 

Example 3.4 It is easy to see that Suzy's throw has degree of responsibility 1/2 for the 
bottle shattering in the naive model described in Figure 1; Suzy's throw becomes critical 
in the contingency where Billy does not throw. In the "better" model of Figure 2, Suzy's 
degree of responsibihty is 1. If we take W to consist of {i?r, BH}, and keep both variables 
at their current setting (that is, consider the contingency where BT — 1 and BH — 0), 
then Suzy's throw becomes critical; if she throws, the bottle shatters, and if she does not 
throw, the bottle docs not shatter (since BH = 0). As we suggested earlier, the setting 
(^r = 0, BT = 1, BH = 0) may be a somewhat unreasonable one to consider, since it 
requires Billy's throw to miss. If the setting {BH — 1, ST — 0, BH — 0) is not allowable, 
then we cannot consider this contingency. In that case, Suzy's degree of responsibility is 
again 1/2, since we must consider the contingency where Billy does not throw. Thus, the 
restriction to allowable settings allows us to capture what seems like a significant intuition 
here. 

As we mentioned in the introduction, in a companion paper [Chockler, Halpern, and 
Kupferman 2003] we apply our notion of responsibility to program verification. The idea 
is to determine the degree of responsibility of the setting of each state for the satisfaction 
of a specification in a given system. For example, given a specification of the form Op 
(eventually p is true), if p is true in only one state of the verified system, then that state 
has degree of responsibility 1 for the specification. On the other hand, if p is true in three 
states, each state only has degree of responsibility 1/3. Experience has shown that if 
there are many states with low degree of responsibility for a specification, then either the 
specification is incomplete (perhaps p really did have to happen three times, in which case 
the specification should have said so), or there is a problem with the system generated by 
the program, since it has redundant states. 

The degree of responsibility can also be used to provide a measure of the degree of 
fault-tolerance in a system. If a component is critical to an outcome, it will have degree of 
responsibility 1. To ensure fault tolerance, we need to make sure that no component has a 
high degree of responsibility for an outcome. Going back to the example of Op, the degree 
of responsibility of 1/3 for a state means that the system is robust to the simultaneous 
failures of at most two states. 
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3.3 Blame 



The definitions of botli causality and responsibility assume that the context and the 
structural equations are given; there is no uncertainty. We are often interested in assigning 
a degree of blame to an action. This assignment depends on the epistemic state of the 
agent before the action was performed. Intuitively, if the agent had no reason to believe, 
before he performed the action, that his action would result in a particular outcome, then 
he should not be held to blame for the outcome (even if in fact his action caused the 
outcome) . 

There are two significant sources of uncertainty for an agent who is contemplating 
performing an action: 

• what the true situation is (that is, what value various variables have); for example, 
a doctor may be uncertain about whether a patient has high blood pressure. 

• how the world works; for example, a doctor may be uncertain about the side effects 
of a given medication; 

In our framework, the "true situation" is determined by the context; "how the world 
works" is determined by the structural equations. Thus, we model an agent's uncertainty 
by a pair (/C, Pr), where /C is a set of pairs of the form (M, u), where M is a causal model 
and M is a context, and Pr is a probability distribution over /C. Following [Halpern and 
Pearl 2001b], who used such epistemic states in the definition of explanation, we call a 
pair (M, u) a situation. 

We think of /C as describing the situations that the agent considers possible before X 
is set to X. The degree of blame that setting X to x has for (p is then the expected degree 
of responsibility of X = x for ip in {Mx^x, u), taken over the situations (M, u) e /C. Note 
that the situation {Mx^x,u) for {M,u) e /C are those that the agent considers possible 
after X is set to x. 

Definition 3.5 (Blame) The degree of blame of setting X tox for cp relative to epistemic 
state (/C, Pr), denoted db(/C, Pr, X <— x, p), is 

dT{iMx^x,u),X^x,ip)PT{{M,u)). 

{M,u)eic 

Example 3.6 Suppose that we are trying to compute the degree of blame of Suzy's 
throwing the rock for the bottle shattering. Suppose that the only causal model that 
Suzy considers possible is essentially like that of Figure 2, with some minor modifications: 
BT can now take on three values, say 0, 1, 2; as before, if BT = then Billy doesn't 
throw, if BT — 1, then Billy does throw, and if BT — 2, then Billy throws extra hard. 
Assume that the causal model is such that if BT — 1, then Suzy's rock will hit the bottle 
first, but if BT = 2, they will hit simultaneously. Thus, SH = 1 if ST— 1, and BH— 1 if 
BT= 1 and SH = Q oi if BT = 2. Call this structural model M. 

At time 0, Suzy considers the following four situations equally likely: 
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• {M,ui), where Ui is such that Billy already threw at time (and hence the bottle 
is shattered); 

• (M, {42), where the bottle was whole before Suzy's throw, and Billy throws extra 
hard, so Billy's throw and Suzy's throw hit the bottle simultaneously (this essentially 
gives the model in Figure 1); 

• (M, ■U3), where the bottle was whole before Suzy's throw, and Suzy's throw hit 
before Billy's throw (this essentially gives the model in Figure 2); and 

• (M, ■U4), where the bottle was whole before Suzy's throw, and Billy did not throw. 

The bottle is already shattered in (M, Ui) before Suzy's action, so Suzy's throw is not a 
cause of the bottle shattering, and her degree of responsibility for the shattered bottle 
is 0. As discussed earher, the degree of responsibility of Suzy's throw for the bottle 
shattering is 1/2 in (M, M2) and 1 in both (M, 1*3) and ((M, {44). Thus, the degree of 
blame is|-| + |- l + |- l = |. If we further require that the contingencies in AC2(b) 
involve only allowable settings, and assume that the setting {ST = 0, BT = 1, BH = 0) is 
not allowable, then the degree of responsibility of Suzy's throw in (M, tTa) is 1/2; in this 
case, the degree of blame is|-| + |- | + |- l = |- 

Example 3.7 Consider again the example of the firing squad with ten excellent marks- 
men. Suppose that marksman 1 knows that exactly one marksman has a live bullet in his 
rifle, and that all the marksmen will shoot. Thus, he considers 10 augmented situations 
possible, depending on who has the bullet. Let pi be his prior probability that marksman 
i has the live bullet. Then the degree of blame of his shot for the death is pi. The degree 
of responsibility is either 1 or 0, depending on whether or not he actually had the live 
bullet. Thus, it is possible for the degree of responsibility to be 1 and the degree of blame 
to be (if he mistakenly ascribes probability to his having the live bullet, when in fact 
he does), and it is possible for the degree of responsibility to be and the degree of blame 
to be 1 (if he mistakenly ascribes probability 1 to his having the bullet when he in fact 
does not). 

Example 3.8 The previous example suggests that both degree of blame and degree of 
responsibility may be relevant in a legal setting. Another issue that is relevant in legal 
settings is whether to consider actual epistemic state or to consider what the epistemic 
state should have been. The former is relevant when considering intent. To see the 
relevance of the latter, consider a patient who dies as a result of being treated by a doctor 
with a particular drug. Assume that the patient died due to the drug's adverse side effects 
on people with high blood pressure and, for simplicity, that this was the only cause of 
death. Suppose that the doctor was not aware of the drug's adverse side effects. (Formally, 
this means that he does not consider possible a situation with a causal model where taking 
the drug causes death.) Then, relative to the doctor's actual epistemic state, the doctor's 
degree of blame will be 0. However, a lawyer might argue in court that the doctor should 
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have known that treatment had adverse side effects for patients with high blood pressure 
(because this is well documented in the literature) and thus should have checked the 
patient's blood pressure. If the doctor had performed this test, he would of course have 
known that the patient had high blood pressure. With respect to the resulting epistemic 
state, the doctor's degree of blame for the death is quite high. Of course, the lawyer's job 
is to convince the court that the latter epistemic state is the appropriate one to consider 
when assigning degree of blame. 

Our definition of blame considers the epistemic state of the agent before the action was 
performed. It is also of interest to consider the expected degree of responsibihty after the 
action was performed. To understand the differences, again consider consider the patient 
who dies as a result of being treated by a doctor with a particular drug. The doctor's 
epistemic state after the patient's death is likely to be quite different from her epistemic 
state before the patient's death. She may still consider it possible that the patient died for 
reasons other than the treatment, but will consider causal structures where the treatment 
was a cause of death more hkely. Thus, the doctor will likely have higher degree of blame 
relative to her epistemic state after the treatment. 

Interestingly, all three epistemic states (the epistemic state that an agent actually 
has before performing an action, the epistemic state that the agent should have had 
before performing the action, and the epistemic state after performing the action) have 
been considered relevant to determining responsibility according to different legal theories 
[Hart and Honore 1985, p. 482]. 

4 The Complexity of Computing Responsibility and 
Blame 

In this section we present complexity results for computing the degree of responsibility 
and blame for general recursive models. 

4.1 The complexity of computing responsibility 

Complexity results for computing causality were presented by Eiter and Lukasiewicz 
[2002a, 2002b]. They showed that the problem of detecting whether X = a; is an ac- 
tual cause of (p is E|'-complete for general recursive models and NP-complete for binary 
models [Eiter and Lukasiewicz 2002b]. (Recall that is the second level of the poly- 
nomial hierarchy and that binary models are ones where all random variables can take 
on exactly two values.) There is a similar gap between the complexity of computing the 
degree of responsibility and blame in general models and in binary models. 

For a complexity class A, FP^'^°^"' consists of all functions that can be computed by 
a polynomial-time Turing machine with an oracle for a problem in A, which on input 
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X asks a total of 0(log|x|) queries (cf. [Papadimitriou 1984]). We show that computing 
the degree of responsibihty of X — x for cp in arbitrary models is FP^^ [^°s"]_gQ]^p]g|^g 
It is shown in [Chockler, Halpern, and Kupferman 2003] that computing the degree of 
responsibility in binary models is FP'^^^f^"^ "'-complete. 

Since there are no known natural FP^^ "'-complete problems, the first step in show- 
ing that computing the degree of responsibihty is FP^2 -complete is to define an 
FP^f ['°s"l-complete problem. We start by defining one that we call MAXQSAT2. 

Recall that a quantified Boolean formula [Stockmeyer 1977] (QBF) has the form 
\/Xi3X2 ■ ■ - ip, where Xi, X2, . . . are propositional variables and -0 is a propositional for- 
mula. A QBF is closed if it has no free propositional variables. TQBF consists of the 
closed QBF formulas that are true. For example, \/X3Y{X ^ Y) E TQBF. As shown 
by Stockmeyer [1977], the following problem QSAT2 is -complete: 

QSAT2 = {3XyYil;{X, Y) e TQBF : e 3-CNF}. 

— * — * 

That is, QSAT2 is the language of all true QBFs of the form 3XyYip, where ^0 is a Boolean 
formula in 3-CNF. 

— * — * — * 

A witness f for a true closed QBF 3XVYijj is an assignment f to X under which 
VYip is true. We define MAXQSAT2 as the problem of computing the maximal number 
of variables in X that can be assigned 1 in a witness for 3X\/YiIj. Formally, given a QBF 
$ = 3X^Yijj, define MAXQSAT2($) to be k if there exists a witness for $ that assigns 

— * 

exactly k of the variables in X the value 1, and every other witness for $ assigns at most 
k' <k variables in X the value 1. If $ ^ QSAT2, then MAXQSAT2($) = -1. 

Theorem 4.1 MAXQSAT2 is FP^^^^°^''^ -complete. 

Proof: First we prove that MAXQSAT2 is in FP^^ [i°g"l Iqj describing an algorithm in 
ppEfiiogn] f^j, solving MAXQSAT2. The algorithm queries an oracle Oc for the language 
£, defined as follows: 

jC^{{^,k) :k>0, MAXQSAT2{^) > k}. 
It is easy to see that £ e Eg : if $ has the form 3XyYijj, guess an assignment / that 

— * 

assigns at least k variables in X the value 1 and check whether / is a witness for Note 
that this amounts to checking the validity of ip with each variable in X replaced by its 
value according to /, so this check is in co-NP, as required. It follows that the language 
C is in E^. (In fact, it is only a slight variant of QSATg.) Given $, the algorithm for 
computing MAXQSAT2($) first checks if $ e QSATg by making a query with A; = 0. If 
it is not, then MAXQSAT2($) is —1. If it is, the algorithm performs a binary search for 
its value, each time dividing the range of possible values by 2 according to the answer of 
Oc- The number of possible values of MAXQSAT2($) is then |X| + 2 (all values between 
— 1 and \X\ are possible), so the algorithm asks Oc at most flogn] -|- 1 queries. 
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Now we show that MAXQSAT2 is FP^^['°s"l-hard by describing a generic reduction 
from a problem in FP^^ MAXQSATg. Let / : {0, 1}* ^ {0, 1}* be a function in 
FP^2 [log"]^ That is, there exists a deterministic oracle Turing machine Mf that on input 
w outputs f{w) for each w e E*, operates in time at most \w\'^ for a constant c > 0, and 
queries an oracle for the language QSAT2 at most dlog \ w\ times (for all sufficiently large 
inputs w) during the execution. We now describe a reduction from / to MAXQSAT2. 
Since this is a reduction between function problems, we have to give two polynomial- 
time functions r and s such that for every input w, we have that r{w) is a QBF and 
s(MAXQSAT2(r(w))) = f{w). 

We start by describing a deterministic polynomial Turing machine that on input w 
computes r{w). On input w of length n, Mr starts by simulating Mf on w. At some step 

^ — * — * 

during the simulation Mf asks its first oracle query qi. The query gi is a QBF 3YiVZi'0i. 
The machine Mr cannot figure out the answer to qi, thus it writes qi down and continues 
with the simulation. Since Mr does not know the answer, it has to simulate the run of 
Mf for both possible answers of the oracle. The machine M,. continues in this fashion, 
keeping track of all possible executions of Mf on w for each sequence of answers to the 
oracle queries. Note that a sequence of answers to oracle queries uniquely characterizes 
a specific execution of Mf (of length n'^). Since there are 2'^'°^" ~ n'^ possible sequences 
of answers, M,. on w has to simulate n possible executions of Mf on w, thus the running 
time of this step is bounded by 0{n'^^'^). 

The set of possible executions of Af/ can be viewed as a tree of height d\ogn (ignoring 
the computation between queries to the oracle). There are at most n'^^^ queries on this 
tree. These queries have the form JYiiZiil^i, for i = 1, . . . , We can assume without 
loss of generality that these formulas involve pairwise disjoint sets of variables. 

Each execution of Mf on w involves at most dlogn queries from this collection. Let 
Xj be a variable that describes the answer to the j'th query in an execution, for 1 < 
j < dlogn. (Of course, which query Xj is the answer to depends on the execution.) 
Each of the n'^ possible assignments to the variables Xi, . . . ,Xdiogn can be thought of 
as representing a number a e {0, . . . , n'' — 1} in binary. Let be the formula that 
characterizes the assignment to these variables corresponding to the number a. That is, 
Ca = Aj=rXf, where Xf is Xj if the jth bit of a is 1 in a and ^Xj otherwise. Under 
the interpretation of Xj as determining the answer the query j, each such assignment a 
determines an execution of Mf. Note that the assignment corresponding to the highest 
number a for which all the queries corresponding to bits of a that are 1 are true is the 
one that corresponds to the actual execution of Mf. For suppose that this is not the 
case. That is, suppose that the actual execution of Mf corresponds to some a' < a. Since 
a' < a, there exists a bit Xj that is 1 in a and is in a'. By our choice of a, the query 
3Y^Ziip corresponding to Xj is a true QBF, thus a -oracle that Mf queries should 
have answered "yes" to this query, and we reach a contradiction. 

The next formula provides the connection between the XiS and the queries. It says 
that if is true, then all the queries corresponding to the bits that are 1 in a must be true 
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QBF. For each assignment a, let Oi, . . . , ajv„ be the queries that were answered "YES" 
during the execution of the Mf corresponding to a. Note that Na < dlogn. Define 

— * — * 

The variables Y^., Za^ do not appear in the formula rj outside of ■0a., thus we can 
re-write r] as 

7] = 3Yi, ...,Y^d,..., VZi, . . . , Z^dT]', 

where 

a=0 

The idea now is to use MAXQSAT2 to compute the highest-numbered assignment a 
such that all the queries corresponding to bits that are 1 in a are true. To do this, it is 
useful to express the number a in unary. Let Ui, . . . , U^d be fresh variables. Note that 
Ui, . . . , Ujid represent a number in unary if there exists 1 < j < n such that Ui = 1 for 
each i < j and Ui = for each i > j. Let ^ express the fact that the UiS represent the 
same number as Xi, . . . , Xaiogn, but in unary: 

i=l a=l 

Let $ = 3t/i,...,[/„d,Xi,...,Xd 

\ogn{v ^ 0* N^t^ that the size of $ is polyiioniial in 
— * — * 

n and, for each i, the variables Yi and Zi do not appear in (f outside of 77. Thus, we can 
rewrite $ in the following form: 

$ = 3Ui, . . . , ?7„d, Xi, . . . , Xdiogn: ^1, • • • , Y^d^Zi, . . . , Z^d{r]' A C). 

Thus, $ has the form 3XyY^jj. 

We are interested in the witness for $ that makes the most f/j's true, since this will 
tell us the execution actually followed by Mf. While MAXQSAT2 gives us the maxi- 
mal number of variables that can be assigned true in a witness for $, this is not quite 
the information we need. Assume for now that we have a machine that is capable of 
computing a slightly generalized version of MAXQSAT2. We denote this version by SUB- 
SET_MAXQSAT2 and define it as a maximal number of I's in a witness computed for 
a given subset of variables (and not for the whole set as MAXQSAT2). Formally, given 
a QBF ^ = 3XVF^ and a set Z C X, we define SUBSET_MAXQSAT2(^, as the 
maximal number of variables from Z assigned 1 in a witness for ^P, or —1 if ^' is not in 
TQBF. 

Clearly, SUBSET_MAXQSAT2($, C/) gives us the assignment which determines the 

execution of Mf. Let r{w) = $. Let s be the function that extracts /(w) from SUB- 
SET _MAXQSAT2($, {f/i, f/n}). (This can be done in polynomial time, since f{w) 
can be computed in polynomial time, given the answers to the oracle.) 
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It remains to show how to reduce SUBSET_MAXQSAT2 to MAXQSAT2. Given a 
formula * = ^XWip and a set f C X, we compute SUBSET_MAXQSAT2(^, f ) in 
the following way. Let U = X \ Z . For each Ui E U we add a variable Let U' 
be the set of all the U- variables. Define the formula \E'' as 3XU'yY{ilj A Auieui^i 
~'Ul)). Clearly, in all consistent assignments to X U U' exactly half of the variables in 

— * — * 

U Li U are assigned 1. Thus, the witness that assigns MAXQSAT2(\1'') variables the 
value 1 also assigns SUBSET_MAXQSAT2(^', Z) variables from Z the value 1. The value 
SUBSET_MAXQSAT2(*, Z) is computed by subtracting \U\ from MAXQSAT2(*')- ^ 

Much like MAXQSAT2, we define MINQSAT2(3XVF^) to be the minimum number 
of variables in X that can be assigned 1 in a witness for 3X\/YiIj if there is such a witness, 
and |X| + 1 otherwise. This problem has the same complexity as MAXQSAT2, as we now 
show. 

Lemma 4.2 MINQSAT2 is FP^^^^''^''^ -complete. 

Proof: For a Boolean formula ijj in 3-CNF, let ip be the same formula, except that each 
propositional variable X is replaced by its negation. It is easy to see that 

MAXQSAT2(3XVyV) = 1^1 - MINQSAT2(3XVyV')- 

Thus, MINQSAT2 and MAXQSAT2 are polynomially reducible to each other and there- 
fore, by Theorem 4.1, MINQSAT2 is FP^^ ['°sn]_^Qj^pig^g □ 

We are now ready to prove FP^^'Pog^l.completeness of computing the degree of respon- 
sibility for general recursive models. 

Theorem 4.3 Computing the degree of responsibility is FF^^^°^'^^ -complete in general 
recursive models. 

Proof: First we prove membership in FP^2 ['og"] i^y describing an algorithm in FP^^ [log"] 
for computing the degree of responsibility. Consider the oracle Oc for the language £, 
defined as follows: 

C^{{{M,u),{X ^x),ip,i) :0>i> 1, a.uddr{{M,u),{X ^x),^p) >i}. 

In other words, {{M,u), {X = x),ip,i) G £ for i > if there is a partition {Z,W) and 
setting (x', w') satisfying condition AC2 in Definition 3.1 such that at most 1/i— 1 variables 
in W have values different in w' from their value in the context u. It is easy to see that 
jC e . Given as input a situation (M, u), X = x (where X is an exogenous variable in M 
and X G TZ{X)), and a formula (f, the algorithm for computing the degree of responsibility 
uses the oracle to perform a binary search on the value of dr{{M,u), {X = x),ip), each 
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time dividing the range of possible values for the degree of responsibility by 2 according 
to the answer of Oc- The number of possible candidates for the degree of responsibility is 
n— |X| + 1, where n is the number of endogenous variables in M . Thus, the algorithm runs 
in linear time (and logarithmic space) in the size of the input and uses at most [log n\ 
oracle queries. Note that the number of oracle queries depends only on the number of 
endogenous variables in the model, and not on any other features of the input. 

The proof that computing the degree of responsibility is FP^2 ['°s"Lhard essentially 
follows from an argument in [Eitcr and Lukasiewicz 2002a] showing that QSAT2 can be 
reduced to the problem of detecting causality. In fact, their argument actually provides a 
reduction from MINQSAT2 to the degree of responsibility. We now describe the reduction 

— * — * 

in more detail. Given as input a closed QBF $ = 3AVS(/7, where </? is a propositional 
formula, define the situation (M, u) as follows. M — {{U ,V ,TZ) , T) , where 

• lA consists of the single variable E\ 

• V consists of the variables A\J B\J {C, X}, where C and X are fresh variables; 

• TZ\s defined so that each variable S & Al) {C, X} has range 7^.(5') = {0, 1}, while 
each variable S E B has range Tl{S) — {0, 1, 2}; 

• Tis defined so that Fg = C + X ior S e B and Fg = ior S e V - B. 

Let u be the context in which E is set to 0. Note that every variable in V has value in 
context u. (This would also be true in the context where E is set to 1; the context plays 
no role here.) Let 

^^{ip' A /\ ,5 ^ 2) V (C = 0) V (X = 1 A C = 1 A y Sy^2), 
seS seS 

where (p' is obtained from (p by replacing each variable SeAUB in iphjS = l. It is 
shown in [Eiter and Lukasiewicz 2002a] that X = is a cause of -0 in (M, «) iff $ e TQBF. 
We now extend their argument to show that the degree of responsibility of X = for 
is actually 1/(MINQSAT2($) + 2). 

Assume first that (X = 0) is a cause of ip in [M,u). Then there exists a partition 
(Z, W) of V and a setting w ofW such that {M,u) \= [X ^ 1,W ^ w]^%Ij and (M, u) |= 
[X ^ 0,W ^ w, Z' ^ 0]ip, where Z' is as in AC2(b). (Since every variable gets value 
in context u, the z* of AC2(b) becomes here.) Suppose that is a minimal set that 
satisfies these conditions. 

— * — * — * — * 

Eiter and Lukasiewicz show that BnW — To prove this, suppose that S e BnW. 

If 5* is set to cither or 1 in w, then {M,u) ^ [W <— w, X <— l]-''*/') since the clause 
[X = 1 A C = 1 A VseB S ^ 2) will be satisfied. On the other hand, if S is set to 2 in w, 
then (Af, u) ^ [W ^ tS, X ^ 0]^. Thus, Br\W = It follows that W <Z AVJ {C}, and 
so Z Q BU {X}. Moreover, C E W, since C has to be set to 1 in order to falsify ip. 
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— * _ 

Note that every variable in W is set to 1 in w. This is obvious for C. If there is some 

— * — * — * 

variable S inW Q A that is set to in w, wc could take S out of W while still satisfying 
condition AC2 in Definition 3.1, since every variable S \nW r\ A has value in context 
u and can only be 1 if it is set to 1 in w. This contradicts the minimality of 1^ . To show 

— * 

that $ e TQBF, consider the truth assignment v that sets a variable S & A true iff S is 

— * — * 

set to 1 in W . It suffices to show that v |= VS(/7. Let v' be any truth assignment that 
agrees with v on the variables in A. We must show that v' \= ip. Let Z' consist of all 
variables S E B such that v'{S) is false. Since {M,u) \= [X ^ (},W ^ 1,Z' ^ 0]iIj, it 
follows that (M, u) \= [X ^ 0,W ^ 1, Z' ^ 0](/9'. It is easy to check that if every variable 
in W is set to 1, X is set to 0, and Z' is set to 0, then every variable in ^4 U -B gets the 
same value as it does in v'. Thus, v' \= ip. It follows that $ e TQBF. This argument also 
shows that there is a a witness for $ (i.e., valuation which makes VS(/7 true) that assigns 

— * — * 

the value true to all the variables in 1^ fl A, and only to these variables. 

Now suppose that if $ G TQBF and let w be a witness for $. Let W consist of C 
together with the variables S E A such that v{S) is true. Let w set all the variables 
in ly to 1. It is easy to see X = is a cause of ip, using this choice of W and w 
for AC2. Clearly {M,u) ^ [X ^ l,W ^ Ij^V', since setting X to 1 and and aU 

— * — * 

variables in 14^ to 1 causes all variables in B to have the value 2. On the other hand, 
(M, u) 1= [X <— 0, <— 1, <— O]'0, since setting X to and W ^ w guarantees that 

all variables in B have the value 1 and tp' is satisfied. It now follows that a minimal set 
W satisfying AC2 consists of C together with a minimal set of variables in A that are 
true in a witness for $. Thus, \W\ = MINQSAT2($) + 1, and we have dr{{M,u), {X = 
0),ip) = 1/{\W\ + 1) = 1/(MINQSAT2($) +2). □ 

4.2 The complexity of computing blame 

Given an epistemic state (/C, Pr), where /C consists of N possible augmented situations, 
each with at most n random variables, the straightforward way to compute db{IC, Pr, X <— 
X, (p) is by computing dr{{M^^^ Y^y' u),X = x,p>) for each (Af, u,Y y) E K. such that 
(M, -u) \= X ^ X /\ and then computing the expected degree of responsibility with 
respect to these situations, as in Definition 3.5. Recall that the degree of responsibility 
in each model M is determined by a binary search, and uses at most [lognMl queries 
to the oracle, where is the number of endogenous variables in M . Since there are N 
augmented situations in /C, we get a polynomial time algorithm with Y^f=i\\ogni\ oracle 
queries. Thus, it is clear that the number of queries is at most the size of the input, 
and is also at most iV[logn*], where n* is the maximum number of endogenous variables 
that appear in any of the N augmented situations in /C. The type of oracle depends on 
whether the models are binary or general. For binary models it is enough to have an NP 
oracle, whereas for general models we need a El'-oracle. 

It follows from the discussion above that the problem of computing the degree of blame 
in general models FP^2 1*^]^ where n is the size of the input. However, the best lower bound 
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we can prove is FP^^'Pog"^]^ by reducing the problem of computing responsibility to that 
of computing blame; indeed, the degree of responsibility can be viewed as a special case 
of the degree of blame with the epistemic state consisting of only one situation. Similarly, 
lower and upper bounds of FP^P[^°§'^1 and FP^^t"! hold for binary models. 

An alternative characterization of the complexity of computing blame can be given 
by considering the complexity classes FP^^ and FPji'^, which consist of all functions that 
can be computed in polynomial time with parallel (i.e., non-adaptive) queries to a 
(respectively, NP) oracle. (For background on these complexity classes see [Jenner and 

Toran 1995; Johnson 1990].) Using FP^^ and FPjj'^, we can get matching upper and 
lower bounds. 

Theorem 4.4 The problem of computing blame in recursive causal models is FP^^ - 
complete. The problem is FP^^^ -complete in binary causal models. 

Proof: As we have observed, the naive algorithm for computing the degree of blame 
uses A'^logn* queries, where N is the number of augmented situations in the epistemic 
state, and n* is the maximum number of variables in each one. However, the answers 
to oracle queries for one situation do not affect the choice of queries for other situations, 
thus queries for different situations can be asked in parallel. The queries for one situation 
are adaptive, however, as shown in [Jenner and Toran 1995], a logarithmic number of 
adaptive queries can be replaced with a polynomial number of non-adaptive queries by 
asking all possible combinations of queries in parallel. Thus, the problem of computing 

blame is in FP^^ for arbitrary recursive models, and in FPlj*^ for binary causal models. 

For hardness in the case of arbitrary recursive models, we provide a reduction from the 

following FP||^ -complete problem [Jenner and Toran 1995; Eiter and Lukasiewicz 2002a]. 

Given N closed QBFs $i, . . . , $iV; where <l>j = ^Ai^Biipi and ipi is a propositional formula 
for 1 < i < iV of size 0{n), compute the vector {vi, . . . ,vn) G {0, 1}^ such that for all 
i e {1, ■ ■ ■ , iV}, we have that Vi — liS^iE TQBF. Without loss of generality, we assume 
that all AiS and BiS are disjoint. We construct an epistemic state (/C, Pr), a formula 
ip, and a variable X such that {vi, . . . ,vn) can be computed in polynomial time from 
the degree of blame of setting X to 1 for (p relative to (/C, Pr). /C consists of the N + 1 
augmented situations (Mj, Ui,(l}), i = 1, . . . , -|- 1. Each of the models Mi, . . . , Mjv+i is 
of size 0{Nn) and involves all the variables that appear in the causal models constructed 
in the hardness part of the proof of Theorem 4.3, together with fresh random variables 
nurrii, . . . , nurriN+i- For 1 < i < N, the equations for the variables in $j in the situation 
{Mi, Ui) at time 1 are the same as in the proof of Theorem 4.3. Thus, $j G TQBF iff X = 1 
is a cause of (pi in (Mj, Ui) at time 1. In addition, for i = 1, . . . , n, in (Mj, Ui) at time 1, the 
equations are such that nurui and nurriN+i are set to 1 and all other variables (i.e., nurrij 
for j ^ {i,Ni} and all the variables in j ^ i) are set to 0; at time 0, the equations are 
such that all variables are set to 0. The situation (Mat+i, un+i) is such that all variables 
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are set to at both times and 1. Let Pr{Mi,Ui) = l/2^(riog"l) for 1 <i < N, and let 
Pr(M7v+i, un+i) = 1 - E^il/2^(riog«l). Finally, let (p = Af=i{numi cpi) A nurriN+i. 

Clearly (/? is in all the N + 1 situations at time (since nurriN^i is false at time 
in all these situations). At time 1, Lp is true in (Mj, -Uj), i = 1, . . . , iV, iff X = 1 is a cause 
of ipi, and if is false in (Mtv+i, mat+i)- Thus, the degree of blame db{)C,PT, X ^ x,ip) is 
an [log n] -bit fraction, where the ith group of bits of size n is not iff $i e TQBF. 
It immediately follows that the vector (f i, . . . , v^) can be extracted from db(}C, Pr, X <— 
X, Lp) by assigning Vi the value 1 iff the bits in the ith group in dh{K,, Pr, X <— x, </?) are 
not all 0. 

We can similarly prove that computing degree of blame in binary models is FPj^^-hard 
by reduction from the problem of determining which of N given propositional formulas 
of size 0{n) are satisfiable. We leave details to the reader. □ 



5 Discussion 

We have introduced definition of responsibility and blame, based on Halpern and Pearl's 
definition of causality. We cannot say that a definition is "right" or "wrong", but we 
can and should examine the how useful a definition is, particularly the extent to which it 
captures our intuitions. 

There has been extensive discussion of causality in the philosophy literature, and many 
examples demonstrating the subtlety of the notion. (These examples have mainly been 
formulated to show problems with various definitions of causality that have been proposed 
in the literature.) Thus, one useful strategy for arguing that a definition of causality is 
useful is to show that it can handle these examples well. This was in fact done for Halpern 
and Pearl's definition of causality (sec [Halpern and Pearl 2001a]). While we were not able 
to find a corresponding body of examples for responsibility and blame in the philosophy 
literature, there is a large body of examples in the legal literature (see, for example, [Hart 
and Honore 1985]). Wc plan to do a more careful analysis of how our framework can be 
used for legal reasoning in future work. For now, we just briefly discuss some relevant 
issues. 

While we believe that "responsibility" and "blame" as we have defined them are impor- 
tant, distinct notions, the words "responsibihty" and "blame" are often used interchange- 
ably in natural language. It is not always obvious which notion is more appropriate in 
any given situation. For example, Shafcr [2001] says that "a child who pokes at a gun's 
trigger out of curiosity will not be held culpable for resulting injury or death". Suppose 
that a child does in fact poke a gun's trigger and, as a result, his father dies. According to 
our definition, the child certainly is a cause of his father's death (his father's not leaving 
the safety catch on might also be a cause, of course), and has degree of responsibility 1. 
However, under reasonable assumptions about his epistemic state, the child might well 
have degree of blame 0. So, although we would say that the child is responsible for his 
father's death, he is not to blame. Shafer talks about the need to take intention into 
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account when assessing culpability. In our definition, we take intention into account to 
some extent by considering the agent's epistemic state. For example, if the child did not 
consider it possible that pulling the trigger would result in his father's death, then surely 
he had no intention of causing his father's death. However, to really capture intention, 
we need a more detailed modeled of motivation and preferences. 

Shafer [2001] also discusses the probabihty of assessing responsibihty in cases such as 
the following. 

Example 5.1 Suppose Joe sprays insecticide on his corn field. It is known that spraying 
insecticide increases the probabihty of catching a cold from 20% to 30%. The cost of a 
cold in terms of pain and lost productivity is $300. Suppose that Joe sprays insecticide 
and, as a result, Glenn catches a cold. What should Joe pay? 

Implicit in the story is a causal model of catching a cold, which is something like the 
following.^ There are four variables: 

• a random variable C (for contact) such that if C = 1 is Glenn is in casual contact 
with someone who has a cold, and otherwise. 

• a random variable / (for immune) such that if / = 2, Glenn does not catch a cold 
even if he both comes in contact with a cold sufferer and lives near a cornfield 
sprayed with insecticide; if / = 1, Glenn does not catch a cold even if comes in 
contact with a cold sufferer, provided he does not live near a cornfield sprayed with 
insecticide; and if / = 0, then Glenn catches a cold if he comes in contact with a 
cold sufferer, whether or not he lives near a cornfield sprayed with insecticide; 

• a random variable S which is 1 if Glenn lives near a cornfield sprayed with insecticide 
and otherwise; 

• a random variable CC which is 1 if Glenn catches a cold and otherwise. 

The causal equations are obvious: CC=liffC=l and either / = or / = 1 and S = 1. 
The numbers suggest that for 70% of the population, / = 2, for 10%, I — 1, and for 
20%, 7 = 0. Suppose that no one (including Glenn) knows whether I is 0, 1, or 2. Thus, 
Glenn's expected loss from a cold is $60 if Joe does not spray, and $90 if he does. The 
difference of $30 is the economic cost to Glenn of Joe spraying (before we know whether 
Glenn actually has a cold). 

As Shafer points out, the law does not allow anyone to sue until there has been damage. 
So consider the situation after Glenn catches a cold. Once Glenn catches a cold, it is clear 
that / must be either or 1. Based on the statisical information, I is twice as hkely to 

be as 1. This leads to an obvious epistemic state, where the causal model where 1 = 
is assigned probability 2/3 and the causal model where / = 1 is assigned probability 1/3. 

^This is not the only reasonable causal model that could correspond to the story, but it is good enough 
for our purposes. 
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In the latter model, Joe's spraying is not a cause of Glenn's catching a cold; in the former 
it is (and has degree of responsibility 1). Thus, Joe's degree of blame for the cold is 1/3. 
This suggests that, once Joe sprays and Glenn catches a cold, the economic damage is 
$100. 

This example also emphasizes the importance of distinguishing between the cpistemic 
states before and after the action is taken, an issue already discussed in Section 3.3. 
Indeed, examining the situation after Glenn caught cold enables us to ascribe probability 
to the situation where Glenn is immune, and thus increases Joe's degree of blame for 
Glenn's cold. 

Example 5.1 is a relatively easy one, since the degree of responsibility is 1. Things can 
quickly get more complicated. Indeed, a great deal of legal theory is devoted to issues of 
responsibility (the classic reference is [Hart and Honore 1985]). There are a number of 
different legal principles that are applied in determining degree of responsibility; some of 
these occasionally conflict (at least, they appear to conflict to a layman!). For example, 
in some cases, legal practice (at least, American legal practice) does not really consider 
degree of responsibility as we have defined it. Consider the rock throwing example, and 
suppose the bottle belongs to Ned and is somewhat valuable; in fact, it is worth $100. 

• Suppose both Suzy and Billy's rock hit the bottle simultaneously, and all it takes is 
one rock to shatter the bottle. Then they are both responsible for the shattering to 

degree 1/2 (and both have degree of blame 1/2 if this model is commonly known). 
Should both have to pay $50? What if they bear different degrees of responsibility? 

Interestingly, a standard legal principle is also that "an individual defendant's re- 
sponsibility docs not decrease just because another wrongdoer was also an actual 
and proximate cause of the injury" (see [Public Trust 2003]). That is, our notion of 
having a degree of responsibility less than one is considered inappropriate in some 
cases in standard tort law, as is the notion of different degrees of responsibility. Note 
that if Billy is broke and Suzy can afford $100, the doctrine of joint and several li- 
ability, also a standard principle in American tort law, rules that Ned can recover 
the full $100 from Suzy. 

• Suppose that instead it requires two rocks to shatter the bottle. Should that case be 
treated any differently? (Recall that, in this case, both Suzy and Billy have degree 
of responsibihty 1.) 

• If Suzy's rock hits first and it requires only one rock to shatter the bottle then, as we 
have seen, Suzy has degree of responsibility or 1/2 (depending on whether we con- 
sider only allowable settings) and Billy has degree of responsibility 0. Nevertheless, 
standard legal practice would probably judge Billy (in part) responsible. 

In some cases, it seems that legal doctrine confounds what we have called cause, blame, 
and responsibility. To take just one example from Hart and Honoree [1985, p. 74], assume 
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that A throws a hghted cigarette into the bracken near a forest and a fire starts. Just as 
the fire is about to go out, B dehberately pours oil on the flame. The fire spreads and 
burns down the forest. Clearly B's action was a cause of the forest fire. Was A's action 
also a cause of the forest fire? According to Hart and Honore, he is not, whether or not 
he intended to cause the fire; only B was. In our framework, it depends on the causal 
model. If B would have started a fire anyway, whether or not A's fire went out, then A is 
indeed not the cause; if B would not have started a fire had he not seen A's fire, then A is 
a cause (as is B), although is degree of resonsibility for the fire is only 1/2. Furthermore, 
A's degree of blame may be quite low. Our framework lets us make distinctions here that 
seem to be relevant for legal reasoning. 

While these examples show that legal reasoning treats responsibility and blame some- 
what differently from the way we do, we believe that a formal analysis of legal reasoning 
using our definitions would be helpful, both in terms of clarifying the applicability of our 
definitions and in terms of clarifying the basis for various legal judgments. As we said, 
we hope to focus more on legal issues in future work. 

Our notion of degree of responsibility focuses on when an action becomes critical. 
Perhaps it may have been better termed a notion of "degree of criticahty". While we 
believe that it is a useful notion, there are cases where a more refined notion may be 
useful. For example, consider a voter who voted for Mr. B in the case of a 1-0 vote and a 
voter who voted for Mr. B in the case of a 100-99 vote. In both case, that voter has degree 
of responsibility 1. While it is true that, in both cases, that voter's vote was critical, in 
the second case, the voter may beheve that his responsibility is more diffuse. We often 
here statements like "Don't just blame me; I wasn't the only one who voted for him!" The 
second author is current working with I. Gilboa on a definition of resonsibility that uses 
the game-theoretic notion of Shapley value (see, for example, [Osborne and Rubinstein 
1994]) to try to distinguish these examples. 

As another example, suppose that one person dumps 900 pounds of garbage on a 
porch and another dumps 100 pounds. The porch can only bear 999 pounds of load, so it 
collapses. Both people here have degree of responsibility 1/2 according to our definition, 
but there is an intuition that suggests that the person who dumped 900 pounds should 
bear greater responsibility. We can easily accommodate this in our framework by putting 
weights on variables. If we use wt{X) to denote the weight of a X and wt(W) to denote the 

— * 

sum of the weights of variables in the set W, then we can define the degree of responsibility 
oi X — X for (p to be wt{X)/{wt{W) + wt{X), where is a set of minimal weight for 
which AC2 holds. This definition agrees with the one we use if the weights of all variables 
are 1, so this can be viewed as a generalization of our current definition. 

These examples show that there is much more to be done in clarifying our understand- 
ing of responsibility and blame. Because these notions are so central in law and morality, 
we believe that doing so is quite worthwhile. 
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